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0, INTRODUCTION 

This paper considers the problem of providing an adequate supply of effec- 
tive manpower when demand for manpower is uncertain and there are time lags in 
the training of effective personnel. The model described in this paper is 
useful in three specific ways: (i) it can calculate optimal manpower input 
policy for alternate control objectives, (ii) it c^>n be used to measure the 
effects of alternate training, retention, and utilization policies on system 
performance, and finally (iii) it can measure the sensitivity of optimal poli- 
cies and system performance to various assumptions on the nature of the sto- 
chastic nature of the demand process. 

In Section 1 we present a model for the flow of personnel. This model 
attempts to capture the essential features of the manpower system in a sitnple 
framework by utilizing the longitudinal stability of various manpower cohorts. 
The stochastic nature of the demand process is described in Section 2. At 
time t , the demand for trained manpower is determined by the state of a 
finite Markov chain. In Section 3 we present two alternate control objec- 
tives. The first objective is to minimize the expected, weighted, discounted 
square of the error between the effective supply and demand for manpower. The 
second objective is to minimize the expected discounted square exror between 
the actual manpower stocks and ideal stocks for any demand state. In Section 4 
we describe th3 form of the optimal policy and optimal value function. Humeri-- 
cal solutions are presented for the two control objectives. In Section 5 we 
outline several uses for the model. The first is the determination of the 
expected manpower stocks when the optimal appointment policy is used. Calcu- 
lation of the equilibrium stocks can then be used to calculate tha expected 
cost of the appointment policy along with some expected measure of system 
performance. These calculations allow us to measure the tvadeoff between 
cost and performance for alternate manpower policies. Ivi Section 5 we also 
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consider the effect of errors in the specification of the stochastic law of 
notion. We find it is relatively easy to discover the effects of these errors. 
Finally in Section 5 we consider the effect of possible alterations in the 
stock of manpower, either by releasing people from the system or allowing 
personnel Inflow that has a past accumulation of experience. 

The appendix is devoted to the theoretical aspect of the model we find 
that the optimization procedure is a hybrid of the linear-quadratic optimal 
control problem ([1] , [4] , [7]) and the Markov decision problem. We are able 
to show that the optimal decision rules are linear and that the optimal policy 
functions are quadratic. In addition, under special assumptions on the prob- 
lem data we can show that the optimal decision rules converge to an equilibrium 
optimal appointment policy. 

The model was motivated by the problem of regulating the supply of naval 
aviators. We do not pretend to present an operational solution to that real 
problem, however, a numerical example, using the naval aviator problem as a 
setting, is used throughout the paper. The author is the sole source of the 
data for the numerical example. 

The paper is based on a longitudinal manpower flow model; Oliver and 
Hopkins [5], [6], that has been used in a similar context by Grinold, Marshall, 
and Oliver, [2], and Grinold and Oliver, [3]. 
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1. THE FLOW OF MANPOI^R 

This section describes the basic manpower flow process. For a more dctiiiled 
description of a related deterministic model see [2] and [3). 

The state of the nanpower system is observed at discrete equa'ly spaced 
time points t « 0,1, 2»3, ...» T ; where T is a planning horizon. The time 
interval (t - l,t] .er time t - 1 and up to and including time t will 
be called pej*iod t . 

At each time t individuals in the system are classified according to 
their length of service (LOS) . The length of service is simply the number of 
occasions the individual has been classified. Thus individuals entering in 
period t will have length of service equal to one at time t , since they 
are in their first period of service at time t . Individuals entering in 
period t - 1 that are still present at time t will be in their second 
period of service. We assume that m is the maximum number of periods an 
individual can remain in the system. It follows that individuals in the 
system at time t in their m'^ period of service must leave the system in 
period t + 1 , 

The stock of manpower at time t is described by the length of service 
distribucion s(t) ; s(t) is a vector with m components, where s. (t) is 

Iv 

the number of individuals at time r with LOS equal to k . 
(1) 8(t) « tSj(t),S2(t), sjt))« 

In each period t a -.lumber of new accessions f(t) enter the manpower 

system. The change in the manpower system from time t to time t + 1 

depends on three factors: the stock at time t » the accessions in period 

t + 1 , and the continuation rates of the stock of manpower at time t . The 

continuation rate q^, for k = 1,2, m - 1 is defined to be the fraction 

_ 

Caution: in [2] and [3] the length of service is defined to be the number of 
completed periods of service, and is then one less than the measure used here. 



of manpower with LOS equal to k » that remains in the system an additional 
period. With this definition we have 

(f (t + 1) if k « 0 
(2) Sj^,(t + 1) . ) 

(qj^Sj^(t) if k « 1,2, m - 1 

The continuation rates are intended to Include several factors that in- 
fluence the flow of loanpower: the organization* s retention, promotion, and 
retirement policy, natural forces such as mortality, and the behavior of 
personnel. 




2» THE DEMAND AND SUPPLY OF KF FECTI.VF. M/\NPOWER 

This section describes the stochastic demand process and the related 
deterministic supply process. 

The demand for effective manpower is a random variable that can 

take on a finite number of values 2^^,22* •••» \ • The stochastic law of 
notion that governs the evolution of the process 2^ is Markovian: 

We assume r^^ is independent of t , and that the uncertain demand 
persists through period t + 1 . 

The example below is a six state system and the manpower category is 
naval aviators. State 1 of the Markov chain represents a calm peacetime 
period, while State 2 represents a higher condition of peacetime preparation. 
States 3, 4, 5, and 6 represent various stages of a conflict. Notice the 
conflict may end, by a return to State 1 or 2, may stay at the same stage or 
it may proceed to a higher stage. The transition probabilities are shown 
below along with the demand for effective manpower and the equilibrium dis- 
tribution of the Markov chain. 

Let i(t) be the state of demand at time t : i.e. 2 ■ z. >. ^ . The 
state of the system at time t depends on two factors: the stock of manpower 
s(t) , and the state of demand, i(t) . 

We shall now generalize the nation of continuation rates introduced in 
(I; 2), by allowing the continuation rates to depend on the state of demand. 
Thus we have continuation rate q^j^ and the law of motion for manpower is 



(3) B^^^it + 1) « 



qj^|.Sj^(t) if k = 1,2, m - 1 
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TABI,E 1 
3 A 



0.90 


0.07 


0.03 








0.5 


0.4 


0.1 








0.05 


0.5 


0.05 


O.A 






O.l 


0.35 




0.2 


0.35 




0.1 


0.6 






0.1 


0.3 


0.1 


0.7 








0.2 



z 
500 
900 
1200 
1200 
1200 
1000 



n 

0.783 
0.U7 
0.G4O 
0.020 
0.008 
0.002 



TABLE 2 



Length of Service 



1 

0.9 


2 
0.9 


3 

0.95 


4 

0.95 


5 

0.95 


6 

0.1 


7 
0.9 


0.9 


0.9 


0.95 


0.95 


0.95 


0.2 


0.9 


0.95 


0.95 


0.9 


0.9 


0.8 


0.8 


0.9 


0.95 


0.9 


0.9 


0.9 


0.9 


0.9 


0.9 


0.95 


0.8 


0.9 


0.8 


0.8 


0.9 


0.9 


0.95 


0.9 


0.9 


0.9 


0.9 


0.9 


0,9 



11 



This allows for alternate retention, mortality, and behavior assumptions for 
each state of the demand process. In this paper wc shall solve numerical 
examples with n »» 6 states and m = 8 maximum years of service. The con- 
tinuation rates are shown in Table 2. 

We can utilize personnel in alternate ways depending on the demand en- 
vironment. Let d^y^ be the contribution to effectiveness of each individual 
in the system with length of service equal to k when the demand environment 
is 1 . The total effectiveness is then 



(A) I a s (t) 

k-l ^ 



and the excess of the supply of effective manpower is given by 



m 

(5) I dik^fcCt) - z. . 

k-1 ^ * 



We shall allow the d^^^ to be nonnegative. Consider the problem of 
supplying effective naval aviators. Individuals In the first period of service, 
are being trained to fly; thus they are not making any contribution to meeting 
the demand for naval aviators. In addition, it takes trained and qualified 
pilots to teach new recruits. Suppose one teacher is needed for every two 
pupils in state i . Then d^^^ » -.5 . Every two pilot trainees take up the 
services of one instructor who is therefore not eligible to meet the demand 
for operational pilots. The contributions effectiveness for the problems 
solved in this paper are seen in Table 3. 
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TABLE 3 
Utilization 
Length of Service 





1 


2 3 i 


\ 5 


6 


7 


8 


1 


"0.5 


"0.5 1 ] 


L 0.8 


0.8 


0.2 


0.2 


2 


"0.5 


"0.5 1 : 




0.8 


0.3 


0.2 


3 


"0.5 


"0.25 1 : 




1 


0.8 


0.8 




"0.5 


"0.3 1 : 




1 


0.8 


0.8 


5 


"0.5 


"0.3 1 : 




1 


0.8 


0.8 


6 


"o.: 


"0.3 1 




1 


0.8 


0.8 
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3> THE CONTROL OBJECTIVE 

This section formulates two quadi-atic objective functions for controlling 
the flow of manpower through the system. We selected quadratic objectives for 
two reasons. First, they lead to ^ model for which results can be readily 
calculated. Second, the objectives are a reasonable measure of the departure 
of actual system performance from an ideal performance. 

The simplest control objective is to minimize the expected weighted 

squared error between the supply and demand for effective manpower. Let 

w. for j » 1,2, n be positive weights that sum to 1. If i(t) * i , 

•J 

and s(t) , then the expected error at time t + 1 , as a function o£ 
f - f (t + 1) , is 



(1) 



n / m-1 V 

[s(t),f(t + 1)] = I r w I z - f(t + 1) - I q.|^s^(t)l 



Our second control objective is to provide a smooth flow of manpower 
through the system and to come as close as possible to meeting the demand 
for effective manpower. If the manpower system was always in state j , 
then an ideal distribution would satisfy 



m 

(2) I d,, s,_ » 2 



and 



s 



k+1 



f if k « 0 



(^jk^k ^"^^f^, ...,m-l 

Note that (2) can be rewritten as 
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if k - 0 



f for k = 1,2, . . . , m - 1 
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For k «= 2,3, m define 



k-1 



and « 1 . 

Coablning (4), (3), and (2) we obtain 

The discrepancy between actual and ideal Is measured by 

k-1 J*^ 
Using the law of motion (II; 3) we can rewrite (7) as 

(8) E*^[s(t),f (t + 1)] « (f(t + 1) - s^^)2 + Y (qikS^(t) - s^^^)2 

This is a measure of the discrepancy at time t + 1 conditioned on four 
facts; manpower s(t) at time t , demand i at time t , accessions f (t + 1) 
in period t + 1 , and demand J at time t + 1 . The function is quadratic, 
strictly convex in f(t + 1) and convex in s(t) . The expected discrepancy 
is 

(9) E^[s(t),f (t + 1)] = I r.,E^^(s(t).f(t + 1)] 

is also quadratic convex in f (strictly) and s . 

We note that it is possible to weigh the functions E^[s(t),f (t + 1)] 
in order to place greater or less importance on the expected discrepancy from 

er|c 15 
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state j . 

For either objective in a T stage problem, where s(t) , and f (t) are 
the observed values of the manpower stocks and accessions for t « 1,2, T , 

and i(t) is the state of demand, the total discounted error is 

T 

(10) I 6^""V^''^[s(t),f(t)] 

t«l 

where 0 < 6 < 1 is a discount factor. The objective is to minimize the 
expected value of (10) over all possible appointment sequences. 
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A. THE OPTIMAL APPOINTMENT POLICY 

This section presents two examples of optimal appointment policies: one 
for each of the objectives presented in Section 3. 

A theoretical description of the optimization procedure is presented in 
the appendix. We shall use three important results from the appendix: 

(i) For any finite planning horizon T , and any demand state i , 
there is an optimal appointment policy that is a linear function 
of the manpower stock s . For our model 

k"»l 

(ii) For any finite planning horizon T , and any demand state i , the 
minimum expected error is a convex quadratic function of the man- 
power stocks s : 

s'P^''^s + q^»^s + r^'^ 

(iii) As the. planning horizon T becomes large, the values of b"''*'^ , 
pi,T ^ qi,T ^ ^i,T ^ converge to b^ , , , r^ . 

In the examples we have solved to date, the convergence has been quite 
rapid. For the first example below a 1% change was observed betvjcen a 20 and 
21 period horizon. In the second example convergence was nearly perfect in 5 
periods. 

The first solution below uses the objective (3:1) with discount factor 
6 = 0.95 and weights 

1/18 if j « 1,2 ) 
^ (4/18 if J = 3,4,5,6) 

17 



The optimal appointment policy is shown in Table 1. 

For the second control objective, (3:9), there is a considerable 

simplification in the optimal decision rule: f » b^ + I b k . . In the 

k=l 

second case b^ « 0 for k > 1 . Thus f •« b^ independently of the curren 
manpower stock s . 

The optimal input as a function of i is shovnx in Table 2. In this 
calculation we set 6 •« 0.95 and did not weigh the various terms in the sum 
(3:9). 
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TABLE 1 



i 


^0 


h 


^2 


S 1 \ 








^8 


1 


667 


'.95 


".17 


".14 


.25 


.30 


0.0 


.10 


0 


2 


604 


~.95 


".16 


".12 


.27 


.31 


.03 


.16 


0 


3 


285 


".97 


.04 


.08 


.37 


.35 


.29 


.39 


0 


4 


105 


".91 


.04 


.21 


.48 


.47 


.40 


.46 


0 


5 


345 


".95 


0.00 


.06 


.36 


.36 


.28 


.32 


0 


6 


475 


".98 


".08 


".03 


.34 


.36 


.22 


.25 


0 



TABLE 2 



X 


^0 


1 


304 


2 


332 


3 


370 


4 


374 


5 


357 


6 


354 
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5. COST PERFORMANCE TRADEOFFS 

This section examines the use of the manpower planning model presented 
in earlier sections to measure the tradeoff between cost and system performance. 
We shall consider this tradeoff in the specific context of the naval aviation 
problem. In particular, we shall assume that costs are determined by the dis- 
tribution, 8(t) , of manpower stocks. 

When an optimal linear appointment policy has been calculated, then it is 
possible to express the evolution of the manpower stocks in a linear fashion 



(1) Sj^j^Ct + 1) « 



**ik\^^^ if k " 1,2 m - 1 

This can be written in matrix form 

(2) s(t + 1) « A^s(t) + h^ , 

The equilibrium expected values of s as a function of the demand state 
j can be calculated. 

Let be the probability that the previous demand state was i given 

that the current demand state is J . These are the inverse transition proba- 
bilities for the Markov chain. When the Markov chain is in equilibrium, a 
simple application of 6ayes*s Law gives 

where ^ " ) is the stationary distribution vector of the Markov 

1 n 

chain. 

Now let s^ for j 1|2 n be the expected nianpower stock in demand 

state j when an optimal linear appointment policy is uf5ed. 
The s*^ must satisfy gQ 
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n 

(A) 



for j » 1,2, n . 

This system of n x m equations with n x m unknowns can be solved for 
the equilibrium expected values of . Tor the first objective (3:1), ir:J 
the appointment policy in Table 1 of Section A, we obtain the equilibrium ex- 
pected values following in Table 1. 

For the second objective (3:9) the expected equilibrium stock levels are 
shown in Table 2. 

With these figures, suitable cost data and the vector ir , it is easy to 
calculate the expected cost per period of operating this manpower system using 
the optimal linear appointment policy. Let c^^ be the cost of an individual 
in the k*"^ period of service. The equilibrium expected cost per period is then 

N. n m J, 

i«l k«l 

We can also construct a measure of system performance* Suppose ve choose 
the expected weighted squared discrepancy between the demand and the steady 
state expected supply level. This would be 

n / m .\2 

Vith a measure of expected system performance and a measure of expected 
cost^ we compare alternate manpower policies. For example, it is possible to 
change the demands and recalculate a new optimal appointment strategy. 

We can also change the continuation rates q^^^ , or the utilization policy 
^ik * ^^^^ these changes would probably produce a corresponding change in the 
costs c^y^ . It is also possible to change the objective function by^ either 

21 



TABLE 1 



i 


1 


2 


3 


4 


5 


6 


7 


8 


1 


371 


332 


298 


283 


268 


254 


29 


30 


2 


350 


325 


296 


279 


262 


243 


73 


57 


3 


362 


328 


297 


282 


267 


251 


42 


41 


4 


285 


329 


308 


270 


251 


216 


199 


66 


5 


376 


279 


289 


276 


240 


223 


195 


179 


6 


328 


348 


241 


252 


222 


194 


195 


175 










TABLE 


2 








1 


1 


2 


3 


4 


5 


6 


7 


8 


1 


307 


279 


252 


241 


229 


217 


25 


26 


2 


330 


295 


262 


243 


226 


209 


63 


49 


3 


318 


285 


257 


243 


229 


215 


36 


35 


4 


371 


312 


273 


234 


217 


186 


171 


57 


5 


373 


353 


281 


246 


209 


192 


168 


153 


6 


357 


351 


289 


254 


203 


171 


169 


151 
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using alternate weights or an entirely different objective function. UsinR 
our model » and trying alternate manpower policies ve can discover, and explore 
the cost-performance tradeoffs that exist. 

There is a second type of sensitivity analysis. If the probabilities in 
the Markov ratrix arc changed, this amounts to an alternate assumption about 
the stochastic nature of the demand proce^^s. By recalculating for several dif- 
ferent values of the transition probabilities we can measure the impact of 
alternate assumptions concerning the demand process. A related question is 
to measure the impact of an Incorrect assumption about the demand process. 
This leads to an easier calculation. Suppose we assume the transition proba- 
bilities are r^^ , when they actually are r^^ with equilibrium distribution 
vector Tt . In this case the appointment policy will not change. However, the 
calculations (3)- (6) would have to use the values r^^ and . In this way 
we can measure the effect of Incorrectly specifying the probabilities. 

In some cases it is possible to alter the manpower stock in an ad-hoc 
fashion. In the language of the naval aviator problem, we can either bring In 
reserves or allow people in the system to leave before their obligated service 
Is completed. If possibilities of this sort exist, then some of the Information 
we have calculated can help in designing an alteration of the manpower stock. 
Recall that the Inimum expected error starting in demand state 1 with stock 
s Is 

(7) sP^s + q^s + r^ . 

The increase in this optimal expected error as a function of an increase in Sj^ 
is simply 

^11 
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This tells hnv an increase or decrease in Sj^ will effect the minimum expected 
error. 

In addition* if the current demand state is i , .then the current manpower 
stock s should be contrasted with its equilibrium value s'^ . It is in the 
interest of system stability to change s closer to its equilibrium value. 

It is even possible to calculate an optimal value for s when the demand 
state is i . Tlie optimum is simply -l/2(P^)~^(q*^)* , the s that minimizes 
(7)* The optimal values are not always realistic* as we indicate below. How~ 
ever, they do indicate a direction in which favorable changes can be made. 

For the objective (3:1) we find the following optimal distributions shown 
in Table 3. Note any value of s^g is optimal. 

The distributions for demand states i « 1,2 are unreasonable, probably 
close to the low veight attached to the objective for those states. The dis- 
tribution for States 3 and 4 is quite good. It indicates that a large increase 
in the three year group would have a favorable impact, as well as a large in- 
crease in the six year group. 

The optimal distributions for the second objective (3:9) are more reasonable. 

Those values shown in Table 4 should be contrasted with those of the equil- 
ibrium distribution in Table 2. 
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TABLK 3 



i 


1 


2 


3 


A 


5 


6 


7 


8 


1 


288 


32 


474 


367 


'16 


7290 


.104 


0 


2 


331 


~61 


481 


554 


131 


2720 


47 


0 


3 


302 


159 


514 


401 


268 


437 


39 


0 




363 


254 


415 


350 


306 


271 


150 


0 


5 


2A2 


244 


583 


398 


137 


369 


"102 


0 


6 


184 


169 


682 


392 


"6 


412 


"201 


0 



TABLE 4 



i 


1 


2 


3 


4 


5 


6 


7 


8 


1 


303 


272 


243 


228 


213 


296 


27 


0 


2 


334 


303 


274 


263 


249 


256 


54 


0 


3 


360 


332 


328 


295 


280 


143 


131 


0 


4 


379 


355 


341 


299 


217 


130 


143 


0 


5 


350 


37? 


323 


342 


274 


91 


98 


0 


6 


342 


330 


318 


305 


266 


80 


83 


0 
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APPENDIX 

DISCRETE TIME, LINEAK-QUADRi\TIC> DYNAMIC PROGRAMS WITH MARKOVIAN PARAMETERS 

This appendix presents, in a more general form, the theoretical results 
used in the paper* Ue employ a notation close to that of the text. 

The state of the system with T + 1 periods remaining is (s,i) • Given 
action f , the state of the system with T periods remaining will be 
(D^(s,f),j) with probability r^^ , where, for each i , D'^(s,f) is a 
linear function of s and f • 

The error associated with choosing f in state (s,i) is E^(s^f) , 
where for each 1 E'^(s,f) is quadratic and convex in s and strictly convex 
in f • Future errors are discounted at rate 6 . 

For i = 1,2, n define V^(s,i) to be the minimum expected discounted 

error for a T period problem that starts with manpower stocks s , and demand 
state i • 

For any choice of f , we have 

(1) Vl^^'^^ - E'-(s.f) + 6 r^^V^vO^(s,f),j) 

By the usual principal of optimality arguments we can establish that the func- 
tions V-,,. (s,i) must satisfy the following functional equation. 




Now assume, that V^(s,i) is a convex quadratic function of s for each 
i • Thus we can write V.^,(s,i) « s^P^'^'^g + q^^'^s + r^*''^ where P^''^ is 
positive semi-definite, and symmetric. It Is easy to show that V^(D^*(s,f ) ,j) 
is a convex quadratic function of (s,f) , and therefore that 
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n . 

(3) E^(s.f) I- 6 I r.,V (D'-(s.f)o) 

is quadratic, convex in s , and strictly convex in f . 

\iiien the quadratic function (3) is differentiated with respect to f , 
then a linear equation results, and the unique f that minimizes (3) as a 
function of s is in the form 

(4) fT+l<s»i) » B^»^^(s) 

i T+1 

where for each 1 and T + 1 » B * (s) is linear in s * When the optltoal 
policy (4) Is substituted Into (3) ve obtain a convex quadratic expression for 
V^^^(s,l) . 

Thus for any finite horizon T the optimal value and policy functions 
are^ respect Ively^ linear and quadratic. We now turn to the question of con- 
vergence of policies as the planning horizon increases. 

Let V^(s,i) « s'P^*^s + q^^'^-s + r^*'''^ be the optimal value function starting 
In state 1 for an I period problem. 

When the discount factor 6 is strictly less than 1» then the sequence 

T 

V (s»l) will remain bounded if there exists a policy f (s,i) such that this 
policy leads to a bounded sequence s(t) for any initial s(o) and any evolu- 
tion of the demand sequence l(t) . In the model described in this paper, the 

policy f (s,i) - 0 will insure s(t) « 0 for t > m . The general question 

T T 

of convergence of V (s,i) and B (s,l) is difficult, and is currently the 
object of further inve itlgation by the author. 
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